Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification

Retrieval augmentation, which enhances downstream models by a knowledge retriever and an external corpus instead of by merely increasing the number of model parameters, has been successfully applied to many natural language processing (NLP) tasks such as text classification, question answering and so on. However, existing methods that separately or asynchronously train the retriever and downstream model mainly due to the non-differentiability between the two parts, usually lead to degraded performance compared to end-to-end joint training. In this paper, we propose Differentiable Retrieval Augmentation via Generative lANguage modeling(Dragan), to address this problem by a novel differentiable reformulation. We demonstrate the effectiveness of our proposed method on a challenging NLP task in e-commerce search, namely query intent classification. Both the experimental results and ablation study show that the proposed method significantly and reasonably improves the state-of-the-art baselines on both offline evaluation and online A/B test.Comment: 5 pages, 2 figures; accepted by CIKM202

Dynamic reorganization of the genome shapes the recombination landscape in meiotic prophase.

In meiotic prophase, chromosomes are organized into compacted loop arrays to promote homolog pairing and recombination. Here, we probe the architecture of the mouse spermatocyte genome in early and late meiotic prophase using chromosome conformation capture (Hi-C). Our data support the established loop array model of meiotic chromosomes, and infer loops averaging 0.8-1.0 megabase pairs (Mb) in early prophase and extending to 1.5-2.0 Mb in late prophase as chromosomes compact and homologs undergo synapsis. Topologically associating domains (TADs) are lost in meiotic prophase, suggesting that assembly of the meiotic chromosome axis alters the activity of chromosome-associated cohesin complexes. While TADs are lost, physically separated A and B compartments are maintained in meiotic prophase. Moreover, meiotic DNA breaks and interhomolog crossovers preferentially form in the gene-dense A compartment, revealing a role for chromatin organization in meiotic recombination. Finally, direct detection of interhomolog contacts genome-wide reveals the structural basis for homolog alignment and juxtaposition by the synaptonemal complex

Common DNA sequence variation influences 3-dimensional conformation of the human genome.

BACKGROUND:The 3-dimensional (3D) conformation of chromatin inside the nucleus is integral to a variety of nuclear processes including transcriptional regulation, DNA replication, and DNA damage repair. Aberrations in 3D chromatin conformation have been implicated in developmental abnormalities and cancer. Despite the importance of 3D chromatin conformation to cellular function and human health, little is known about how 3D chromatin conformation varies in the human population, or whether DNA sequence variation between individuals influences 3D chromatin conformation. RESULTS:To address these questions, we perform Hi-C on lymphoblastoid cell lines from 20 individuals. We identify thousands of regions across the genome where 3D chromatin conformation varies between individuals and find that this variation is often accompanied by variation in gene expression, histone modifications, and transcription factor binding. Moreover, we find that DNA sequence variation influences several features of 3D chromatin conformation including loop strength, contact insulation, contact directionality, and density of local cis contacts. We map hundreds of quantitative trait loci associated with 3D chromatin features and find evidence that some of these same variants are associated at modest levels with other molecular phenotypes as well as complex disease risk. CONCLUSION:Our results demonstrate that common DNA sequence variants can influence 3D chromatin conformation, pointing to a more pervasive role for 3D chromatin conformation in human phenotypic variation than previously recognized

A compendium of chromatin contact maps reveals spatially active regions in the human genome

The three-dimensional configuration of DNA is integral to all nuclear processes in eukaryotes, yet our knowledge of the chromosome architecture is still limited. Genome-wide chromosome conformation capture studies have uncovered features of chromatin organization in cultured cells, but genome architecture in human tissues has yet to be explored. Here, we report the most comprehensive survey to date of chromatin organization in human tissues. Through integrative analysis of chromatin contact maps in 21 primary human tissues and cell types, we find topologically associating domains highly conserved in different tissues. We also discover genomic regions that exhibit unusually high levels of local chromatin interactions. These frequently interacting regions (FIREs) are enriched for super-enhancers and are near tissue specifically expressed genes. They display strong tissue-specificity in local chromatin interactions. Additionally, FIRE formation is partially dependent on CTCF and the Cohesin complex. We further show that FIREs can help annotate the function of non-coding sequence variants

Transcriptionally active HERV-H retrotransposons demarcate topologically associating domains in human pluripotent stem cells.

Chromatin architecture has been implicated in cell type-specific gene regulatory programs, yet how chromatin remodels during development remains to be fully elucidated. Here, by interrogating chromatin reorganization during human pluripotent stem cell (hPSC) differentiation, we discover a role for the primate-specific endogenous retrotransposon human endogenous retrovirus subfamily H (HERV-H) in creating topologically associating domains (TADs) in hPSCs. Deleting these HERV-H elements eliminates their corresponding TAD boundaries and reduces the transcription of upstream genes, while de novo insertion of HERV-H elements can introduce new TAD boundaries. The ability of HERV-H to create TAD boundaries depends on high transcription, as transcriptional repression of HERV-H elements prevents the formation of boundaries. This ability is not limited to hPSCs, as these actively transcribed HERV-H elements and their corresponding TAD boundaries also appear in pluripotent stem cells from other hominids but not in more distantly related species lacking HERV-H elements. Overall, our results provide direct evidence for retrotransposons in actively shaping cell type- and species-specific chromatin architecture

Type 1 diabetes risk genes mediate pancreatic beta cell survival in response to proinflammatory cytokines

Publisher Copyright: © 2022We combined functional genomics and human genetics to investigate processes that affect type 1 diabetes (T1D) risk by mediating beta cell survival in response to proinflammatory cytokines. We mapped 38,931 cytokine-responsive candidate cis-regulatory elements (cCREs) in beta cells using ATAC-seq and snATAC-seq and linked them to target genes using co-accessibility and HiChIP. Using a genome-wide CRISPR screen in EndoC-βH1 cells, we identified 867 genes affecting cytokine-induced survival, and genes promoting survival and up-regulated in cytokines were enriched at T1D risk loci. Using SNP-SELEX, we identified 2,229 variants in cytokine-responsive cCREs altering transcription factor (TF) binding, and variants altering binding of TFs regulating stress, inflammation, and apoptosis were enriched for T1D risk. At the 16p13 locus, a fine-mapped T1D variant altering TF binding in a cytokine-induced cCRE interacted with SOCS1, which promoted survival in cytokine exposure. Our findings reveal processes and genes acting in beta cells during inflammation that modulate T1D risk.Peer reviewe

Multi-tissue integrative analysis of personal epigenomes

Evaluating the impact of genetic variants on transcriptional regulation is a central goal in biological science that has been constrained by reliance on a single reference genome. To address this, we constructed phased, diploid genomes for four cadaveric donors (using long-read sequencing) and systematically charted noncoding regulatory elements and transcriptional activity across more than 25 tissues from these donors. Integrative analysis revealed over a million variants with allele-specific activity, coordinated, locus-scale allelic imbalances, and structural variants impacting proximal chromatin structure. We relate the personal genome analysis to the ENCODE encyclopedia, annotating allele- and tissue-specific elements that are strongly enriched for variants impacting expression and disease phenotypes. These experimental and statistical approaches, and the corresponding EN-TEx resource, provide a framework for personalized functional genomics

Interpreting human genetic variations through transcriptional regulation and 3D genome organization

It has been more than a decade since the human genome was sequenced, but a complete understanding of the functional elements in the human genome is still lacking, especially for the non-coding part of the genome. The lack of complete understanding of the genome makes interpreting the function of genetic variants a daunting challenge. Here I exploited multiple ways to decipher the function of genetic variants by leveraging knowledge about transcriptional regulation and three-dimension genome organization. First, we developed SNP-SELEX, a high throughput method to assess the effect of SNPs on transcription factor (TF) binding. I demonstrated the superior performance of SNP-SELEX over previous delta PWM models, and applied results of SNP-SELEX to identify putative causal variants for type 2 diabetes. Furthermore, I employed deltaSVM algorithm to develop models that could predict the effect of SNPs on TF binding for any non-coding variants. Those models not only outperform delta PWM models in vitro and in vivo but also could help identify novel master regulator for complex traits and diseases.Next, I co-led a study to investigate the effect of genetic variants on three-dimensional (3D) chromatin conformation. I identified thousands of regions across the genome where 3D chromatin conformation varies between individuals and found those variations often accompany changes in other genome functions. Moreover, I found DNA sequence variations could influence 3D chromatin conformation and mapped hundreds of Quantitative Trait Loci (QTLs) associated with 3D chromatin features, some of which confer disease risk. Finally, I analyzed Hi-C data from human embryonic stem cells differentiated to beta cell progenitors to characterize changes in chromatin organizations during differentiation. I identified chromatin loops that are dynamic during different stages and found those loops are also associated with transcriptional regulation. Further, I revealed that chromatin loops form interaction hubs that are related to the establishment of stage-specific transcriptional programs

Interpreting human genetic variations through transcriptional regulation and 3D genome organization

It has been more than a decade since the human genome was sequenced, but a complete understanding of the functional elements in the human genome is still lacking, especially for the non-coding part of the genome. The lack of complete understanding of the genome makes interpreting the function of genetic variants a daunting challenge. Here I exploited multiple ways to decipher the function of genetic variants by leveraging knowledge about transcriptional regulation and three-dimension genome organization. First, we developed SNP-SELEX, a high throughput method to assess the effect of SNPs on transcription factor (TF) binding. I demonstrated the superior performance of SNP-SELEX over previous delta PWM models, and applied results of SNP-SELEX to identify putative causal variants for type 2 diabetes. Furthermore, I employed deltaSVM algorithm to develop models that could predict the effect of SNPs on TF binding for any non-coding variants. Those models not only outperform delta PWM models in vitro and in vivo but also could help identify novel master regulator for complex traits and diseases.Next, I co-led a study to investigate the effect of genetic variants on three-dimensional (3D) chromatin conformation. I identified thousands of regions across the genome where 3D chromatin conformation varies between individuals and found those variations often accompany changes in other genome functions. Moreover, I found DNA sequence variations could influence 3D chromatin conformation and mapped hundreds of Quantitative Trait Loci (QTLs) associated with 3D chromatin features, some of which confer disease risk. Finally, I analyzed Hi-C data from human embryonic stem cells differentiated to beta cell progenitors to characterize changes in chromatin organizations during differentiation. I identified chromatin loops that are dynamic during different stages and found those loops are also associated with transcriptional regulation. Further, I revealed that chromatin loops form interaction hubs that are related to the establishment of stage-specific transcriptional programs